Efficient computer-implemented techniques for managing graphics memory

ABSTRACT

In an embodiment, a data processing method provides an improvement in efficient use of computer memory and comprises using a computer, creating in computer memory a glyph memory area that is configured to store a plurality of cached glyphs; using the computer, receiving a request from an application to use a particular glyph; in response to the request, determining whether the particular glyph is in the glyph memory area; in response to determining that the particular glyph is not in the glyph memory area: attempting to store a bitmap of the particular glyph to a next location in the glyph memory area; in response to determining that the next location is not available a first time, reclaiming space in the glyph memory area in an amount sufficient to store the bitmap; attempting a second time to store the bitmap in the next location in the glyph memory area; in response to determining that the next location is not available a second time, clearing the glyph memory area of all previously stored glyphs and storing the bitmap in the glyph memory area.

BENEFIT CLAIM

This application claims the benefit under 35 USC §119(e) of provisionalapplications 62/119,746, filed Feb. 23, 2015, and 62/119,747, filed Feb.23, 2015, the entire contents of which are hereby incorporated byreference for all purposes as if fully set forth herein. Theapplicant(s) hereby rescind any disclaimer of claim scope in theprovisional applications and advise the USPTO that the claims in thisapplication may be broader than any claim in the provisionalapplications.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to computer-implementedtechniques for manipulating and displaying digitally stored graphicalimages. The disclosure relates more specifically to glyph caching,surface caching, surface atlases, and efficient display of large numbersof graphical images.

BACKGROUND

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection.

Computer services, applications and systems that are configured todisplay large numbers of graphical images to large numbers of clientcomputers must offer high performance in order to deliver the best userexperience. For example, certain kinds of audiovisual program servicesare configured to display graphical images representing movies, TVshows, or other items that are available for viewing. When theseservices offer thousands of items, and have millions of subscribers,efficient techniques for delivering millions of copies of the graphicalimages to client computers are needed.

A related problem involves displaying electronic documents, web pages orother graphical content that include text formed using stored graphicalelements termed glyphs. A glyph may be a character or pictogram of analphabet or character set that can be used to form words, phrases orsentences. Although the total number of glyphs is typically small—forexample, a standard Roman alphabet has 26 letters that may correspond to26 glyphs—online systems of the type described above may need to deliverparticular glyphs millions of times to different clients as part ofcomputer displays that the systems generate. Thus, there is a need fortechniques to efficiently deliver glyphs from computer storage to thedisplay units of the client computers.

SUMMARY

The appended claims may serve as a summary of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings:

FIG. 1 illustrates a networked computer system with which an embodimentmay be implemented.

FIG. 2 illustrates data structures, memory organization and otheraspects of glyph caching according to an embodiment.

FIG. 3 illustrates a process of managing a glyph cache, in oneembodiment.

FIG. 4 illustrates an example computer memory space configured to storea plurality of atlases of graphical surfaces.

FIG. 5 illustrates data structures that may be used to supportcomputer-implemented surface caching.

FIG. 6 illustrates a gallery memory space having sets of pre-loadedimage elements and a viewport.

FIG. 7 is a three-part diagram that illustrates the viewport of FIG. 6in states in which all image elements in visible rows are displayed, afirst portion of image elements are undisplayed and represented bygrayed-out polygons, and a second portion of image elements areundisplayed and represented by grayed-out polygons.

FIG. 8 illustrates a process of controlling the display of images in agraphical user interface display.

FIG. 9 illustrates an example graphical user interface with which thetechniques of FIG. 8 may be used.

FIG. 10 illustrates a computer system with which an embodiment may beimplemented.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however,that the present invention may be practiced without these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order to avoid unnecessarily obscuring thepresent invention.

Embodiments are described in sections below according to the followingoutline:

1. Overview

2. Glyph Caching Techniques

3. Surface Atlasing Techniques

4. Surface Caching Techniques

5. Image Display and Pre-Fetch Techniques

6. Example Implementation Mechanism—Hardware Overview

7. Extensions and Alternatives

1. Overview

A computer application, such as a media player, may request textualcharacters, graphical surfaces, images, textures or other visual contentto be displayed on a screen of a computer on which the application runs.To do so, the application may invoke calls, through an operating systemor directly, to one or more processing units to convert the requestedvisual content item into a bitmap which, perhaps after additionalprocessing, can be written to the screen of the computer and cause adisplay of the visual content item. In order to accelerate theconversion of a visual content item into a corresponding bitmap, in anembodiment, the computer creates and maintains a memory space forbitmaps and metadata of visual content items. Accordingly, once aparticular visual content item is stored as a bitmap in the memoryspace, when the application requests the display of the particularvisual content item, the computer may directly retrieve the bitmapstored in the memory space, rather than download, decode, transform,render and/or perform other visual content processing operations for theparticular visual content item. To maximize advantages of storing visualcontent items in memory spaces, various techniques are described hereinto efficiently store and quickly access visual content items from amemory space. These techniques may result in increase of theresponsiveness of the application and decrease computing resourcesconsumed to display visual content items.

In an embodiment, a data processing method provides an improvement inefficient use of computer memory and comprises using a computer,creating in computer memory a glyph memory area that is configured tostore a plurality of cached glyphs; using the computer, receiving arequest from an application to use a particular glyph; in response tothe request, determining whether the particular glyph is in the glyphmemory area; in response to determining that the particular glyph is notin the glyph memory area: attempting to store a bitmap of the particularglyph to a next location in the glyph memory area; in response todetermining that the next location is not available a first time,reclaiming space in the glyph memory area in an amount sufficient tostore the bitmap; attempting a second time to store the bitmap in thenext location in the glyph memory area; in response to determining thatthe next location is not available a second time, clearing the glyphmemory area of all previously stored glyphs and storing the bitmap inthe glyph memory area.

In an embodiment, a data processing method comprises using a computer,creating and storing, in computer memory, one or more first rows ofvisible image elements and one or more second rows of non-visible imageelements; causing displaying, in a graphical user interface, graphicalimages corresponding to the one or more first rows of visible imageelements; receiving input specifying a scrolling operation that involvesmoving at least a portion of one of the second rows of non-visible imageelements into visible position; causing displaying animated movement ofthe one or more first rows of visible images and the one or more secondrows based upon the input; during the displaying the animated movement,for the non-visible image elements, refraining from performing one ormore of downloading, decoding, rendering or displaying imagescorresponding to the non-visible image elements, and displaying one ormore placeholder images in newly visible positions corresponding to thenon-visible image elements; detecting a pause in the input specifying ascrolling operation; in response to the detecting, performing one ormore of downloading, decoding, rendering or displaying rendered imagescorresponding to the non-visible image elements, and displaying therendered images in the newly visible positions; wherein the method isperformed by one or more special-purpose computing devices.

FIG. 1 illustrates a networked computer system with which an embodimentmay be implemented. In one approach, an application server computer 102is coupled to a network 104, which is also coupled to a content deliverynetwork (CDN) 106 and client computers 110, 130, 140. For purposes ofillustrating a clear example, FIG. 1 shows a limited number of elements,but in practical embodiments there may be any number of certain elementsshown in FIG. 1. For example, application server 102 may represent aninstance among a large plurality of instances of the application serverin a data center, cloud computing environment, or other mass computingenvironment. There may be multiple CDNs and there may be thousands ormillions of client computers.

In an embodiment, application server computer 102 hosts a media catalog103 that contains metadata relating to a plurality of movies, TVprograms, or other audiovisual items that may be delivered by streamingor otherwise to the client computers 110, 130, 140 from CDN 106. The CDN106 comprises a plurality of content servers 108, 109 that hold copiesof assets associated with the audiovisual items, such as digitizedversions of the audiovisual items prepared using a variety of differentencodings and/or formats. Network 104 broadly represents one or morelocal area networks, wide area networks, internetworks, or a combinationthereof.

Each of the client computers 110, 130, 140 comprises a computing devicehaving a central processing unit (CPU), graphics processing unit (GPU),one or more buses, memory organized as volatile and/or nonvolatilestorage, one or more data input devices, I/O interfaces and outputdevices such as loudspeakers or a LINE-OUT jack and associated drivers.Each of the client computers 110, 130, 140 is coupled to a separatedisplay unit 112 such as a computer screen, TV screen or other display.Client computers 110, 130, 140 may comprise any of mobile or stationarycomputers including desktop computers, laptops, netbooks, ultrabooks,tablet computers, smartphones, etc. Typically the GPU and CPU eachmanage separate hardware memory spaces. For example, CPU memory may beused primarily for storing program instructions and data associated withapplication programs, whereas GPU memory may have a high-speed busconnection to the GPU and may be directly mapped to row/column driversor driver circuits associated with a liquid crystal display (LCD) thatserves as the display 112.

Each of the client computers 110, 130, 140 hosts, in an embodiment, amedia player application 114 running on an operating system 118 andcomprising glyph caching logic 120, surface caching logic 122, surfaceatlasing logic 124, and image pre-fetch logic 126. In an embodiment,media player application 114 is configured to establish a connection toapplication server computer 102, access the media catalog 103, andprovide a display of graphical images representing available and/orrecommended audiovisual items to the client computer 110. In anembodiment, application server 102 may maintain a plurality of accounts,each associated with one of the client computers 110, 130, 140 and/orone or more users of the client computers. Accounts may include profilesof multiple different users in a household who use the same clientcomputer 110, 130, 140 to access the application server computer 102.

In this arrangement, client computer 110 may select a particularaudiovisual item for viewing, receive from the application servercomputer 102 a location identifier of a particular content server 108that hosts copies of assets for the particular audiovisual item, requestthe particular audiovisual item from the particular content server, andreceive streaming media delivery of one of the assets for display on thedisplay unit 112. Thus, the media player application 114 on clientcomputer 110 serves as a means of accessing the media catalog 103 andassets held on the content servers 108, 109; the media playerapplication may integrate many features and functions, such asrecommendations, lists of queued items to play, social interaction withfriends of a user of the client computer, display of metadata relatingto audiovisual items, and many other functions other than only playingaudiovisual items.

In an embodiment, the glyph caching logic 120 is configured to supportoperation of the media player application 114 by efficiently obtaining,managing and displaying glyphs as part of the displays that the mediaplayer application generates, using the techniques further describedherein. In an embodiment, the surface caching logic 122 is configured tosupport operation of the media player application 114 by efficientlyobtaining, managing and displaying graphic images that are organized assurfaces using the techniques further described herein; examples ofsurfaces include images representing particular audiovisual items,sometimes termed box art. In an embodiment, surface atlasing logic 124is configured to support operation of the media player application 114by efficiently storing and managing the storage of surfaces, using thetechniques further described herein. In an embodiment, image pre-fetchlogic 126 is configured to support operation of the media playerapplication 114 by efficiently displaying box art or other images aspart of media catalog displays, particular during scrolling or otherdisplay operations, using the techniques further described herein.

In one approach, the media player application 114 may be viewed as aclient of the glyph caching logic 120, surface caching logic 122,surface atlasing logic 124, and image pre-fetch logic 126. For example,the glyph caching logic 120, surface caching logic 122, surface atlasinglogic 124, and image pre-fetch logic 126 may be implemented asindependent callable methods or processes that provide services, and themedia player application 114 may be principally responsible forgenerating a graphical user interface on the client computer 110 andcalling services as needed from the logical elements.

Further, glyph caching logic 120, surface caching logic 122, surfaceatlasing logic 124, and image pre-fetch logic 126 may be supported by agraphic image processing library and/or font library, which may beimplemented using commercial products or open source projects. Forexample, in one embodiment the client computer 110 also hosts animplementation of the FREETYPE project, a view system such as OPENGL,and/or other libraries or projects, which implement graphics primitiveoperations and/or interact directly with a graphics processing unit(GPU) of client computer 110. For purposes of illustrating clearexamples, certain sections herein refer to using FREETYPE calls andOPENGL structures, but other embodiments may interoperate in afunctionally similar manner with other font systems and graphicsprimitives.

2. Glyph Caching Techniques

FIG. 2 illustrates data structures, memory organization and otheraspects of glyph caching according to an embodiment. FIG. 3 illustratesa process of managing a glyph cache, in one embodiment.

Referring first to FIG. 2, in an embodiment, computer memory at clientcomputer 110 may be organized using a glyph cache 208 that is coupled toor accessible using the glyph caching logic 120. In an embodiment, glyphcaching logic 120 also is coupled to geometry data 212 and a glyphvector 210, which may be implemented using data structures in computermemory of client computer 110. In general, the glyph cache 208 comprisesa finite memory space that is used to store copies of a finite number ofglyphs as the media player application 114 requests the glyphs fordisplay as part of GUI elements of displays that the applicationgenerates. Thus, the glyph cache 208 may serve as an atlas of glyphs foruse by requesting applications. Preferably the glyph cache 208 isallocated in memory that is managed by the GPU, rather than by thecentral processing unit (CPU) of the client computer 110.

The glyph vector 210 comprises a stored list of bitmaps of all glyphsthat are defined for a particular character set; for example, glyphs202, 204, 206 may comprise bitmaps of different characters of acharacter set. Glyph vector 210 may comprise vector format data for afont or characters within a font and may include other data necessaryfor rendering text into glyphs for display; for example, TRUETYPE fontdata may be used.

Geometry data 212 comprises a plurality of glyph items 214, 216, each ofwhich comprises a set of metadata for a particular glyph 202, 204, 206and is indexed using a distinct key 220. Large numbers of keys, such askey 220, may be cached in a dedicated cache for fast access to metadataabout glyphs. For purposes of illustrating a clear example, FIG. 2 showstwo glyph items 214, 216 but in a practical embodiment there may be anynumber of glyph items in the geometry data 212. Further, to show a clearexample, FIG. 2 shows geometry data 212 and glyph cache 208 as separateunits, but in some embodiments, both the geometry data and glyphs may becached in a single memory address space and managed as a unified cache;this approach would permit, for example, rapid access to baseline dataand other metadata contained in glyph items 214, 216. As a specificexample, obtaining width and height data could be obtained from thecache without requiring a FREETYPE call to access the vector 210 orother stored font data. Alternatively, even if a FREETYPE call isissued, the FREETYPE operation to render text may result in access toglyph cache 208 rather than re-loading and re-processing TRUETYPE fontdata from persistent storage. In one approach, when glyphs 202, 204, 206are added to or removed from cache 208, geometry data 212 forcorresponding glyph items 214, 216 is concurrently updated in the samememory space; that is, glyphs and related data may stand or fall in thecache together. Alternatively, metadata and glyphs are independent andthe cache may store geometry data 212 for a particular glyph 202, 204,206 without having that glyph in the cache 208.

For purposes of illustrating a clear example, FIG. 2 shows a singleglyph cache 208. However, there may be a plurality of glyph caches 208,each cache corresponding to a particular character set or font family.

In one embodiment, each glyph item 214, 216 comprises values for: glyphidentifier, which may correspond to key 220; a distance above baseline;a distance below baseline; a width; a height; an upper left address; aused/not used flag; alpha channel values. Glyph items, such as glyphitem 214, may include bitmaps of the corresponding glyphs in addition oralternative to the glyphs being stored in glyph cache 208. In anembodiment, the distance above baseline value indicates a number ofpixels that the associated glyph extends above a baseline that isdefined for the font to which the glyph belongs, and the distance belowbaseline value is a number of pixels that the glyph extends below thebaseline. As an example, in a typical Roman character set the letter “A”has a distance below baseline value of zero (or a small number) sincethe letter is displayed at or just above the baseline, whereas theletter “g” has a non-zero value for distance below baseline because thedescender portion is displayed below the baseline of all othercharacters. The width value indicates a width of the glyph in pixels andeach glyph may have a different width value. The height value indicatesa height of the glyph in pixels and each glyph may have a differentheight value.

The upper left address value indicates a memory location in the glyphcache 208 at which an upper left pixel of the glyph bitmap data begins.Additionally or alternatively, the system may maintain a separatemapping of keys to memory locations. Accordingly, it is unnecessary totraverse geometry data 212 to access glyphs in glyph cache 208, ratherthe mapping of keys to memory locations may be used to access theglyphs.

The used/not used flag indicates whether the associated glyph iscurrently used in a GUI display of the media player application 114; asfurther described, a value of NOT USED may be used as the basis ofdefragmentation, deletion, or other memory recapture or re-useoperations associated with glyph cache 208. The alpha channel value maybe from 0 to 1 and represent opacity of the glyph and may also includeanti-aliasing data.

A usage index 240 may be associated with glyph cache 208 and may storecache occupancy or availability data, or a combination, indicating whichlocations in the cache are occupied by glyphs and/or which locations areavailable to accept new glyphs. For example, usage index 240 maycomprise a plurality of entries, each entry indicating a range of bytesthat are in use for a glyph, and/or a set of upper-left address valuesin association with height values and width values. In effect the usageindex 240 may comprise a series of pointers to in-use portions of thecache 208. The particular structure or format of data in the usage index240 is not critical provided that the index provides a way to rapidlydetermine what space in the glyph cache 208 is actually occupied orfree, without having to access data in the cache itself.

The foregoing format of metadata is merely one example, and fonts orglyphs used on different hardware platforms comprising the clientcomputers 110, 130, 140 may used different sets of metadata and/ordifferent data formats. For example, metadata might include RGBA data,full pixel fonts, etc.

Glyph caching logic 120 may implement the process of FIG. 3. In anembodiment, at block 302, the process establishes a memory area forstoring a plurality of cached glyphs. For example, the glyph cache 208is allocated in memory using a memory allocation operation of theoperating system 118 and a specified size. In one implementation, glyphcache 208 is 1024 bytes by 1024 bytes, but other sizes may be used inother embodiments. This size will accommodate a large number of glyphs,which typically comprise bitmaps of roughly 30×30 bytes for non-Asianalphabets. The size of the glyph cache 208 may be a configurable valuethat can be changed based upon a configuration file or other setting.Note that the memory within the glyph cache 208 is initially empty andis not pre-populated before applications need to use glyphs.

At block 304, the process receives a request from an application to showa particular glyph. For example, the media player application 114 sendsa request to glyph caching logic 120 to display a particular characterin a particular font. Block 304, followed by other blocks of FIG. 3, areintended to clarify that the process of FIG. 3 is dynamic and performedon-demand in response to applications that need to display fonts,characters, strings, phrases or sentences, so that the glyph cache 208is not prepared in advance for later use by the applications.

At block 305, the process tests whether the requested glyph is alreadyin the glyph cache. Evaluation of the test of block 305 may involveperforming a lookup in the geometry data 212 to determine if the glyphis in use. If so, then at block 310 the process performs a graphicsoperation for the glyph. Block 310 broadly represents instructing theGPU to blit the glyph to the screen; performing colorization operationbased on the alpha channel data in the geometry data 212 and colorvalues obtained from the vector 210 or other sources; instructing theGPU to perform a direct memory copy from the glyph cache 208 to alocation in display memory that the application has specified; returningthe bitmap of the glyph from the cache to the requesting application; orother operations. In some cases, as a result, when a requested glyph isin the cache, the glyph can be immediately displayed without atransformation operation, because the transformation operation wasalready performed to place the glyph in the cache.

If the test of block 305 is negative, then in response, at block 306,the process causes transforming glyph vector data into a bitmap. Forexample, block 306 may involve issuing a FREETYPE call to causetransforming vector data for glyph 202 into a bitmap representation. Forpurposes of clarity, FIG. 3 omits certain preparatory steps that may beused as a matter of completeness in an implementation. For example, inone embodiment, if necessary block 306 or other blocks within FIG. 3 mayinclude requesting loading a TRUETYPE font and requesting the font or afont library to return a set of glyphs for a certain font size. Thecomputational cost of performing block 306 is relatively high, andtherefore embodiments herein are configured to minimize the number oftimes that such vector transformations to bitmaps are needed.

At block 308, the process attempts to add the resulting bitmapdynamically to the glyph cache in the next available location.Attempting to add a bitmap to the cache may comprise inspecting theusage index 240 to determine which locations in cache 208 are availableor occupied. At block 312, the process tests whether any of theavailable locations will accept the glyph.

In an embodiment, the location of the subject glyph is determined basedon properties of recently added glyphs. The test of block 312 may beperformed by comparing width and height values of the subject glyph towidth and height values of two or more glyphs that were most recentlyadded to the cache. For example, assume that glyph 202 was added tocache 208 as seen in FIG. 2. If the next glyph 204 to be added has awidth greater than the width of glyph 202, then the next glyph 204 isadded to the right of glyph 202. However, if the next glyph 206 to beadded has a height greater than the height of glyph 202, then the nextglyph 206 is added directly below glyph 202. This approach maximizesefficient use of memory. These logical rules may be implemented in apacker method or object that is called from the glyph caching logic 120,or may be implemented within that logic.

In some embodiments, the next available location is selected to leave a1-pixel gap between adjacent glyphs, which has been found beneficial inthe case of rendering using OPENGL in which rounding error and otherinaccuracies in floating-point operations require such padding space toprevent font overlaps. Implementations that do not use OPENGL may notneed to use such padding.

If a location is available using the foregoing logic, then at block 314,the glyph is added in that location. Geometry data 212 also is updatedto reflect the location at which the corresponding glyph was actuallystored by updating the upper left address value.

However, the foregoing logic may result in determining that the correctnext position for the current glyph is not available in cache 208because the cache already holds too many other glyphs. If this occurs,then in block 316, the process tests whether the failure to find anavailable location is the first or second failure. If it is the firstfailure, then in block 318, the process reclaims a portion of space inthe cache. In one embodiment, reclaiming space comprises deleting allglyphs that are in a specified amount of the cache 208, such as the 30%of the memory that comprises the cache and is associated with UNUSEDglyphs in the geometry data 212. The value 30% is not critical and othervalues or amounts may be used in other embodiments.

Additionally or alternatively, reclaiming space may comprise performingdefragmentation of the cache 208 by rewriting certain glyphs into newpositions that have become available after removal of non-used glyphs.For example, rather than deleting everything in the lower 30% of memory,the reclamation operation may comprise deleting only those glyphs thatare in the cache 208 but marked as UNUSED in the geometry data 212, thenrepacking the USED glyphs into new positions based upon availability.Additionally or alternatively, rather than relying upon the USED/UNUSEDflag in the geometry data 212, the process may maintain a list of glyphidentifiers in order of most recently used or priority, and may use thelist to determine which glyphs to delete from the cache or which memoryareas to reclaim.

If the test of block 316 is true so that the failure to find anavailable location is the second failure, then at block 320, the processcompletely clears the cache and then stores the subject glyph in thefirst available location, as represented in FIG. 3 by the path fromblock 320 to block 314. That is, when an iteration fails for the secondtime, the process clears the entire cache and starts rebuilding thecache as the application requests the use of glyphs.

In one embodiment, if a particular glyph is to be rendered using avisible outline, the outline is treated as a separate glyph, so that theoutline is requested from the font, transformed to a separate bitmap,and stored in glyph cache 208 in a position separate from a characterthat is to be outlined. In other words, outlined characters are notstored in unitary fashion in the glyph cache 208, but a plain characteris stored in one location in the cache and an outline for the characteris stored in another location.

Using these approaches, the glyph cache 208 is constructed andmaintained dynamically in response to requests of applications todisplay glyphs, rather than in advance. The dynamic constructionapproach described here is beneficial in cases in which it is notpossible to predict in advance which character set will be used, orwhich characters within that set will be needed for an application torender a dynamically generated computer screen display. In theapproaches herein, requests of applications to cause displaying glyphscan be satisfied by fast memory copy operations that move a portion ofthe cache 208 corresponding to a glyph bitmap from GPU memory to GPUregisters for display processing; repeated invocation of transformationsfrom vector data to bitmaps are avoided, greatly improving computationaland processing efficiency. This approach precludes the need torepeatedly convert font elements from vector format to bitmaps in everyframe of rendering, and also avoids relatively costly memory copyoperations that move data from CPU memory to GPU memory. Theseapproaches therefore are expected to reduce the time needed to displayglyphs and related words, phrases or sentences in high-speedgraphics-heavy applications that serve large numbers of clientcomputers.

Still another benefit is that the present approaches can reduce thenumber of calls to GPU primitive operations that need to be performed.For many GPUs, a single function call can be used to perform operationsfor multiple glyphs when the source location is the same. In the presentapproaches, the glyph cache 208 serves as a single source that can bereferenced in a single call to process multiple different glyphs. Forexample, assume that the application needs to render the word HELLO onthe display screen and the cache 208 contains glyphs for H, E, L, O. Asingle GPU function call may be used, referencing the cache 208 once andthe four (4) locations within cache 208 of the glyphs for H, E, L, L, O,to request the GPU to batch blit all five (5) characters contiguously tothe screen. In contrast, with some prior approaches, batch blitoperations could not have been performed and five (5) blit calls couldhave been required, each referencing a different source for theconstituent glyphs.

As a further optimization, in implementations that use OPENGL, thesystem may cache an OPENGL vertex buffer 230 that contains locations,within glyph cache 208, of sets of glyphs that are expected to bereused. As one example, in FIG. 2 vertex buffer 230 contains locationdata 232, 234, 236 corresponding to locations in glyph cache 208 forglyphs 202, 204, 206. As another example, a cached OPENGL vertex buffercould contain location data in glyph cache 208 for the sequence ofglyphs H, E, L, L, O. Thereafter, when the application requestsrendering the word HELLO to the screen again, for example, the systemmay provide the GPU with a single instruction that references theparticular vertex buffer that contains glyph location data for HELLO. Inresponse, the GPU will directly access the glyph cache 208 iterativelyfor each glyph identified in the vertex buffer and blit each particularglyph to the screen. This approach can eliminate the need to form andsend the GPU a plurality of different blit instructions for individualcharacters of a word or phrase.

3. Surface Atlasing Techniques

FIG. 4 illustrates an example computer memory space configured to storea plurality of atlases of graphical surfaces. In an embodiment, theelements of FIG. 4 are created and managed by the surface atlasing logic124 in the manner described herein in subsequent sections.

In the example of FIG. 4, a memory space 401 comprises a plurality ofmemory units 402 each representing an array of bytes corresponding to agraphical image having fixed dimensions of length and width. In oneapproach, the techniques of FIG. 4 are optimized for use with graphicalimages that are rectangular with predictable or known dimensions, suchas graphical images of box art pertaining to audiovisual recordings.Thus each memory unit 402 typically comprises complete bitmap dataincluding color information for a particular item of box art or anotherkind of graphical image.

In the example of FIG. 4, memory space 401 comprises rows 404, 406, 408each having five (5) memory units 402. In an embodiment, each of therows 404, 406, 408 corresponds to all the box art images that aredisplayed in a particular row of a graphical user interface on thedisplay device of an end user. Particular examples of such GUI screensare described further herein in other sections.

In an embodiment, the surface atlasing logic 124 of media playerapplication 114 may declare one or more visual content items, such asgraphical images, and associated memory units 402 as part of aparticular surface atlas group. Thus the memory units 402 may beorganized as a plurality of surface atlas groups. Each surface atlasgroup, or “atlas,” may be stored in a finite memory size of fixed andspecified size, such as the same size as for the glyph cache describedherein for FIG. 2. In an embodiment, media player application 114 maydeclare the fixed size of the graphical images and the size of eachatlas. GPU may treat an atlas group as a single image for visual contentprocessing operations, significantly reducing computing resourcesnecessary for executing the operations. For example, in an embodimentthat is based on OPENGL, each atlas is implemented as a texture.

The surface atlas groups may or may not correspond to rows of the memoryspace 401 and therefore may or may not correspond to rows of images thatare capable of display in a row-oriented user interface. For example, afirst surface atlas group 410 comprises four (4) memory units identifiedwith number “1”, a second surface atlas group has five (5) unitsidentified with number “2”, and a third surface atlas group has two (2)units identified with number “3”. Metadata indicating an identifier of asurface atlas group associated with an image may be stored in geometrydata of the type described herein for glyphs. Alternatively, anembodiment may maintain an LRU list of surfaces, ordered by leastrecently used (LRU) memory unit, and the atlas identifier may be storedin an entry of the LRU list along with data identifying the LRU unit.

Memory organized in this manner permits more rapid graphical transferoperations, uses less memory by implementing efficient reclamation andreferencing approaches, and reduces the number of texture operationsthat would be required otherwise. For example, in response to the mediaplayer application 114 requesting to display a row of box art images ina GUI that is configured to show recommendations, lists, queues or otherassociations of audiovisual programs, the surface atlasing logic 124 maybe configured to batch blit all memory units 402 of a particular surfaceatlas group 410 to the display screen. Thus, a single GPU operation canbe used with a reference to a single memory space 401, and the four (4)units 402 within a particular surface atlas group 410 in that space,greatly reducing the number of GPU cycles required to write all theunits of a surface atlas group 410 to the screen. Moreover, to supportthe appearance of scrolling operations in the GUI, the surface atlasinglogic 124 may issue animation instructions to the GPU that reference thesingle memory space and a group of memory units 402 that correspond tothe particular surface atlas group 410; as a result, fast scrolloperations may be permitted in an environment in which users typicallyregularly and rapidly scroll large lists of graphical items around thescreen as part of catalog browsing operations. For example, aninstruction may instruct the GPU to offset all items in the surfaceatlas group 410 by “20” pixels and in response the GPU will perform abatch memory copy operation to accomplish the shift. Repeated use ofsuch operations can produce animated effects and scrolling.

In some embodiments, the elements of a surface atlas group 410 maycorrespond to particular rows or visible rows of the GUI. When allelements of a first surface atlas group 410 are presently displayed onthe screen of the end user GUI and the media player application 114needs to add other images to visible positions of the GUI, the surfaceatlasing logic 124 may add the new images to the same surface atlasgroup 410 having label “1”. A surface atlas group 410 may expanddynamically in this manner until the specified memory size for thesurface atlas group is full. At that point, if the media playerapplication 114 requests adding another image to the visible portion ofthe GUI and all other images must be retained because the other imagesare also visible, in response, the surface atlasing logic 124 may form anew surface atlas group having a different surface atlas groupidentifier, such as “2” in the example of FIG. 4. Forming a new surfaceatlas group may require a new memory allocation operation to allocatememory in the specified amount for use for the new surface atlas group.

Similarly, if a particular image in a particular surface atlas groupenters a non-visible region of display space, then the surface atlasinglogic 124 may remove that image from the surface atlas group 410. Thesurface atlasing logic 124 also may implement reclamation logic that istriggered based on various thresholds. For example, in one embodimentwhen the surface atlasing logic 124 determines that 20% or less of thememory space of a particular surface atlas group is associated withimages that are then currently visible on screen, the surface atlasinglogic may purge the entire surface atlas group in order to reclaim thememory space associated with the surface atlas group. If the remainingimages in that surface atlas group are needed for display then thesurface atlasing logic 124 may rebuild a new surface atlas group forthose images as they are required by the media player application 114.In other embodiments, a threshold for reclaiming memory from an atlasgroup may be based on a particular number or total size of non-visibleimages in the surface atlas. The threshold may be configurable by anapplication such as a media player.

In an embodiment, surface atlasing logic 124 implements duplicate imagedetection logic. For example, assume that a particular memory unit 412associated with surface atlas group “1” is for the graphic image fileFOO.png. Assume now that media player application 114 performsoperations that request using the same graphic image FOO.png as part ofa different surface atlas group “2.” In response, surface atlasing logicis configured not to retrieve a second copy of FOO.png from networkedstorage or to request transformation to a bitmap, since a bitmap forthat image was already obtained when the image was associated withsurface atlas group “1”. Instead, geometry data or other storage mayreflect a reference to the prior bitmap of FOO.png. This procedure canimprove the efficiency of retrieval, storage and display operations whena particular image is requested multiple times for display in the GUI.

In a related procedure, in an embodiment, surface atlasing logic 124 isconfigured to manage multiple atlases to handle orphan references tographic images. For example, assume now that all the images of surfaceatlas group “1” have scrolled off-screen in the GUI, so that surfaceatlas group “2” remains valid with visible images, but FOO.png is nolonger displayed in the position that was associated with surface atlasgroup “1”. In this case, the only reason for retaining storage ofsurface atlas group “1” is to prevent the loss of image data that isreferenced in surface atlas group “2”. In this case, in response,surface atlasing logic 124 is configured to transfer the bitmap data forFOO.png to surface atlas group “2” and to reclaim all storage associatedwith atlas group “1”. This approach ensures that the system does notwaste storage maintaining surface atlas groups that are associatedentirely with non-displayed images, yet hold source data that wasretrieved for the benefit of surface atlas groups and therefore retainsusefulness. As a result, far more GPU memory is available for otheruses.

4. Surface Caching Techniques

FIG. 5 illustrates data structures that may be used to supportcomputer-implemented surface caching. In an embodiment, the surfacecache 401 may be an LRU list 502 that comprises data identifying aplurality of surface items 501 in least recently used order. Any numberof surfaces may be associated in the LRU list 502. For example, a firstsurface item in the LRU list may comprise surface 504 which comprisegraphical image bitmap data 506, a surface atlas group ID 508, aUSED/UNUSED flag 510 and a network location value 512. The graphicalimage bitmap data 506 may comprise a rendered texture that is availableto blit to a screen for use by an application; for example a surfacemight comprise box art for the program “Hunger Games.” Surface atlasgroup ID 508 identifies a surface atlas group to which the item belongs,as described in the previous section. The USED/UNUSED flag 510 indicateswhether the item is then currently in-use by an application, forexample, displayed in a screen display in a visible position, and servesas an indicator about whether the item should be retained or reclaimed,as further described herein. The network location value 512 indicates alocation at which a source file for the rendered texture may be found;for example, the location may be indicated using a URL that includes apath that terminates with a filename for the image such as FOO.png.

In an embodiment, as surfaces are rendered to the GUI, the surfaces arereordered in LRU list 502. The surfaces may be re-ordered by updatingLRU list entries' sequence identifiers that indicate the positions ofthe LRU entries according to the order of the LRU list.

In one particular implementation, the LRU list 502 has finite size basedupon the platform, GPU, and other constraints and could use 25 MB ofmemory, for example; other embodiments may use different memory spacesizes for the LRU list. Therefore, operation of the GUI or otherapplications typically will eventually cause the LRU list 502 to fill,or approach a full state, at the time that a new surface needs to beadded to the LRU list. Therefore, in an embodiment, surface cachinglogic 122 is configured to inspect the LRU list 502 in least recentlyused order to determine whether the list contains one or more old itemshaving a size that is equivalent to the new surface to be added. Forexample, if the new surface to be added is 100×100 pixels, then thesurface caching logic 122 searches the LRU list 502 until either anexisting old 100×100 surface is found or one or more surfaces occupyingtotal memory that exceeds 100×100 is found.

If the LRU list 502 contains a surface item that was not recently usedand exactly matches the size of the new surface to be added, then thenew surface to be added replaces the old item in the LRU list andassociated metadata is updated. Therefore, previously allocated memorywith a handle and assigned to the GPU may be re-used, reducing the needto inform and use the GPU to destroy a space and create a new space. Forexample, if a URL referencing BAR.png is received, and that image hasthe same dimensions as FOO.png, which is in the cache but not recentlyused, then the cache entry for FOO.png may be cleared and BAR.png may berendered into a bitmap in the same memory previously allocated for FOO.In effect, the memory is pooled for use with multiple images withoutde-allocation or complex reclamation operations.

In some cases, the USED/UNUSED flag 510 of an LRU list item 504 may notaccurately reflect the actual state of use of an item. This can occur,for example, when an application separate from the surface caching logic122 requests the use of a surface, so that the surface is added to LRUlist 502, but later updates the screen display so that the surface isnot visible and does not inform the surface caching logic that thesurface is effectively not in use. Therefore, in an embodiment, thesurface caching logic 122 also is configured to query requestingapplications and/or their data structures and/or supporting view systemsto determine whether a particular surface actually is in use and toupdate the USED/UNUSED flag 510 and/or to purge a surface from the LRUlist 502 if the querying determines that the surface is not actually inuse by the application.

The network location 512 of an LRU list item 504 may be checked inresponse to an application requesting the retrieval, use or rendering ofa particular graphic image in the application. For example, the surfacecaching logic 122 may be configured to receive requests fromapplications to display particular images, where the requests include aURL of a specified image. In response, surface caching logic 122 checksLRU list 502 based upon values of network location 512 of all items inthe list, to determine if a network location value matches the URLrequested by the application. If so, then the bitmap data 506 for thematching LRU list item may be used to satisfy the request of theapplication. If there is no match, then the surface caching logic 122 isconfigured to determine if a file matching the URL has been downloadedbut not yet decoded and rendered. If so, then the surface caching logic122 may initiate decoding and rendering the downloaded file to yield thebitmap data 506 and create a new LRU list item 504 subject to thereclamation policies and methods described above. However, ifdownloading has not occurred, then the surface caching logic 122 isconfigured to initiate a network transaction to download the filespecified in the URL, decode the file, render the bitmap data 506, andcreate a new LRU list item 504 subject to the reclamation policies andmethods described above.

In some embodiments, the reclamation processes described herein areperformed only when the cache or LRU list has become nearly full, andnot before.

Each file may be also associated with a lazy flag value. In thiscontext, lazy operation refers to performing fewer than all, or none, ofthe operations involved in obtaining a graphical image file andpresenting the graphical image on screen, including downloading,decoding, rendering, and causing displaying. If the lazy flag value isnot set for a particular graphical image file, then in response to arequest from an application to obtain the particular graphical imagefile, the surface caching logic 122 is configured to perform thedownloading, decoding and rendering operations, even if the image is notin a visible location of the screen. Thereafter, if the applicationperforms a scroll operation that causes the image to move to a visiblelocation of the screen, the surface caching logic 122 may blit thepreviously rendered bitmap using the GPU immediately. In an embodiment,in response to a request of an application to use a particular graphicalimage file that is associated with a first lazy flag value, the surfacecaching logic 122 is configured to store a marker indicating that theparticular graphical image file has been requested, without performingany of the downloading, decoding and rendering operations; thereafter,if the application performs a scroll operation that causes the image tomove to a visible location of the screen, the surface caching logic 122is configured to perform the downloading, decoding, rendering andblitting operations.

In an embodiment, in response to a request of an application to use aparticular graphical image file that is associated with a second lazyflag value, the surface caching logic 122 is configured to download thespecified file using a network transaction, store the file in the cachein original format without forming a surface, for example as a PNG file;thereafter, if the application performs a scroll operation that causesthe image to move to a visible location of the screen, the surfacecaching logic 122 is configured to perform the decoding, rendering andblitting operations.

In some embodiments, images also may be associated with a hardwareacceleration flag value with which an application mandates that GPUhardware acceleration is used to render an image. For example, in someembodiments the media player application 114 may request the renderingof a specified graphical image and may require GPU hardware accelerationfor the rendering, as part of the request. This approach is appropriatewhen the media player application 114 is configured to displayparticular kinds of images or images of a particular size or locationfor which hardware acceleration is necessary to obtain a good userexperience. In other words, including a mandate for hardwareacceleration can provide improved performance when the media playerapplication 114 is aware that the image to be displayed has a size orother characteristics for which hardware acceleration will be needed toprovide appropriate performance.

These approaches effectively decouple the operations to download,decode, and render assets, such as visual content items, to enableefficiently selecting when to perform particular ones of the operations,unlike prior approaches in which all the operations are bound andrequired to be performed serially together. Decoupling the operationsenables providing application developers with the ability to control howto use them by setting values of the lazy flag. The caching logic alsoenforces execution priority rules regardless of the lazy value; forexample, rendering of visual content items that are known to beoff-screen may be deferred even if the lazy value instructs immediaterendering, so that a finite number of decode operations, for example, isdone at a time and the next operation is selected from a dynamicallysorted list. Thus, for visual content items that are not visible onscreen in the UI of the user device, the processes herein can deferrendering while the user device is idle, and resume rendering when theuser resumes interacting with the device.

5. Image Display and Pre-Fetch Techniques

In an embodiment, a user interface application that displays a catalogof audiovisual program items may support forming user lists of titlesthat may include hundreds of items. In this environment, there may beinsufficient memory and resources to load box art images for all theitems in one or more user lists. Therefore, in an embodiment a viewportis configured that includes the visible screen area and regions justabove and below the visible screen area. Gallery display logic isconfigured to fetch box art images only for audiovisual program itemsthat are within the viewport, and is configured to pre-fetch box artimages based on a schedule determined according to user interaction. Forexample, the gallery display logic loads and renders images foraudiovisual program items that are in a user list but conceptually justoutside the viewport, so that the user might scroll to those images inthe near future. Further, embodiments are configured to reuse the samespecified number of memory elements to hold or represent many moreimages by intelligently filling and replacing the memory elements withimages according to a variety of policies. Prefetching may occur forboth sides of the viewport (left and right) as well as for entire rowsthat are conceptually above or below the visible area of the viewport.

Embodiments are described with reference to FIG. 1, FIG. 6, FIG. 7. FIG.6 illustrates a gallery memory space having sets of pre-loaded imageelements and a viewport. While certain embodiments are described hereinin the context of displaying box art images for movies or TV programs,the techniques described herein apply generally to collections of anykind of item. FIG. 7 is a three-part diagram that illustrates theviewport of FIG. 6 in states in which all image elements in visible rowsare displayed, a first portion of image elements are undisplayed andrepresented by grayed-out polygons, and a second portion of imageelements are undisplayed and represented by grayed-out polygons. As seenin FIG. 1, in an embodiment, in client computer 110 the media playerapplication also may comprise gallery display logic 128 that isconfigured to perform the functions that are further described in thissection and coupled to active image elements 168 in memory andassociated with an image element pool 172.

Referring now to FIG. 6, in an embodiment, a gallery memory space 602may comprise a set of pre-loaded image elements organized in a pluralityof rows denoted A, B, C, D, E, F. Each row A, B, C, D, E, F comprises afinite number of image elements, such as image elements 607, 608, 609 inthe case of row A and image elements 611, 610, 612 for row B. Forpurposes of illustrating a clear example, the embodiment of FIG. 6includes six (6) rows A, B, C, D, E, F, but other embodiments may haveany number of rows and the specific number of rows may be configurablein code that implements player application 114, in a profile associatedwith client computer 110, or in other configuration data. For example,there could be 15 rows. Similarly, for purposes of illustrating a clearexample, the embodiment of FIG. 6 includes six (6) rows image elementsin each of the rows A, B, C, D, E, F, but other embodiments may have anynumber of image elements per row and the specific number of imageelements per row may be configurable in code that implements playerapplication 114, in a profile associated with client computer 110, or inother configuration data. Thus, the size of pre-loaded image elements604 in total may vary across different platforms.

Two rows A, B have some image elements 608, 610 within a viewport 606that denotes a portion of the image elements and memory that arecurrently visible in the user interface of a media player application.For example, the gallery display logic 128 may be configured to generatea graphical user interface that includes a search box, GUI widgetsconfigured to enable selection of display options, and a plurality ofrows of box art images indicating audiovisual program items, where eachof the rows is associated with a genre, type of list such as recentviews or recommendations, or other metadata. These rows of box artimages may be scrollable in response to user input of a pointing deviceor from touch-screen gestures that indicate moving a particular row leftor right, or scrolling all visible rows up or down.

In such an example environment, when the media player application 114and gallery display logic 128 initiate operation, rows A, B may bevisible and therefore image elements 608, 610 are visible on the screen.All other rows C, D, E, F lie outside the viewport 606 that the gallerydisplay logic 128 manages and therefore box art images of those rows arenot visible on screen; moreover, image elements 607, 609, 611, 612 areoutside the viewport and not visible, although the image elements arepart of rows A, B that are partly visible. For purposes of illustratinga clear example, FIG. 6 shows only two image elements 607, 609 that areoutside the viewport for row A and two image elements 611, 612 that areoutside the viewport for row B; in embodiments, however, any number ofsuch image elements may be within a row and outside the viewport. Forexample, a row A, B could consist of 20 image elements of which 10 arein the viewport 606 and 10 are outside the viewport. When operation ofthe application 114 initiates, the gallery display logic 128 requestsone or more of the other logical elements of the application to fetch,download, and render those images that are in rows A, B, and to blit theimages of image elements 608, 610 to cause displaying them on thescreen.

Input indicating scrolling row A left or right may cause image elements609 or 607 to enter the viewport 606 and become visible, respectivelywhile dragging or gesturing row B left or right may cause image elements612, 611 to become visible. Each of the rows A, B, C, D, E, F may beorganized as a circular buffer. For example, an operation that moves rowA to the left may result in moving image element 609 into the viewportso that is visible; continuing the same operation would cause movingimage element 607 into the viewport at the next position to the right ofimage element 609, because in a circular buffer arrangement the imageelement 607 is logically adjacent to the right of element 609. Circularscrolling may be subject to constraints; for example, in one embodimentif a list or row comprises ten or fewer items, circular scrolling is notimplemented and lateral scrolling of that row terminates at the lastitem to the left or right rather than wrapping around.

Input or touch gestures that indicate scrolling all rows down may resultin moving row F into the viewport, moving row A down one position, andmoving row B out of the view port and thus off screen. Alternativelyinput indicating scrolling all rows up may result in moving row A intothe position of row F, moving row B up, and moving rows C, D, E upwardinto successively higher positions.

FIG. 8 illustrates a process of controlling the display of images in agraphical user interface display. FIG. 9 illustrates an examplegraphical user interface with which the techniques of FIG. 8 may beused.

At block 802, the process determines a plurality of lists of items; eachlist corresponds to a category. For example, in the context of the mediaplayer application 114, the process could establish a RECENTLY PLAYEDlist that includes all audiovisual programs that the then-currentlylogged-in user has recently played; a COMEDIES list that includes aplurality of audiovisual programs in the media catalog 103 that are inthe genre COMEDIES; and other lists as appropriate for the application,user or context. The plurality of lists typically is established in mainmemory of the client computer 110, and the row structure 620 asdescribed further below may be used as a mechanism for storing andmanaging one of the lists.

At block 804, the process associates a first set of one or more of thelists with one or more rows of at least some visible image elements thatare stored in computer memory, and also associates a second set of oneor more of the lists with one or more rows of non-visible image elementsin computer memory. For example two (2) rows may be associated with atleast some visible image elements that are within a viewport as well asimage elements that are just outside the viewport and in the same row,whereas a second set of lists may be completely outside the viewport andassociated with stored image elements in non-visible locations. As aresult, some of the lists that were established at block 802 areassociated with entirely non-visible image elements and rows, whileothers are associated with rows of image elements that are at leastpartly within the viewport and visible.

At block 806, the process displays one or more rows of visible images.For example, block 806 may represent initial execution of a media playerapplication, gallery viewing application, or other application thatpresents rows of digital graphical images, where the application isconfigured to enable each of the rows to be scrolled left or right, andto enable all the rows to be scrolled up or down.

At block 808, the process receives input specifying a vertical scrollingoperation that requires moving at least one of the rows of non-visibleimage elements into visible position. For example, touch gesture input,cursor control input, mouse input or other input is received indicatingscrolling a set of rows or the entire screen display up or down. Theinput also may be for moving a particular row left or right so thatcertain non-visible image elements of that row are moved into theviewport.

At block 810, the process causes displaying animated movement of the oneor more rows of visible images and one or more new rows based upon theinput. For example, referring again to FIG. 6, FIG. 7, block 810comprises causing animated movement of rows A, B upward and causinganimated movement of rows C, D upward so that at least row C enters theviewport 606. However, as seen at block 812, during the displaying, theprocess refrains from immediately fetching, downloading, decoding,rendering or displaying images in positions that are newly visible, anddisplays a placeholder image in those positions. The placeholder imagemay be, for example, a gray box, a transparent blank box, a black box,etc. As seen in FIG. 7 view (B) and view (C) for example, performingblock 810, 812 may result in displaying one or more rows in which atleast one or more graphical images are represented by the placeholderimages and actual images, such as box art, are not shown.

At block 814, a test is performed to determine whether the scrollinginput has paused. If not, then control returns to block 808. The loop ofblock 814 to 808 represents repeatedly receiving vertical scrollinginput over a period of time. For example, a looping operation may beassociated with a user rapidly scrolling the screen or using repeatedgenerally vertical touch gestures, trackball input indicating a verticaldirection, thumbwheel input on a mouse indicating vertical scrolling,etc.

At block 816, if the test of block 814 is positive, then in response tothe pause that has been detected, the process performs the fetching,downloading, decoding, rendering and/or displaying for a specifiednumber of the images that are in positions that are newly visible. Assuggested by the term “and/or,” one or more such operations may havebeen performed previously as part of the surface atlasing and/or surfacecaching techniques that have been previously described. Further, block816 may include fetching metadata, and not just image data, associatedwith images; metadata may include, for example, an image name or title,synopses or other descriptive information about an audiovisual programthat is represented in the image, location identifiers such as URLs forbox art images, date-time information, etc. Thus, block 816 broadlyrepresents performing whatever image retrieval, decoding, rendering ordisplaying operations that are needed to cause obtaining and displayingon the screen an image corresponding to an image display element thathas moved into the viewport 606 with any associated metadata that may beuseful to the application or needed for display.

After block 816, control may return to block 806 or 808; the loop fromblock 816 to 806, 808 generally indicates that operation of the processmay be continuous and may respond to scrolling input or pausing as theyoccur. An infinite loop is not implied and when the process of FIG. 8 isintegrated into a larger application the application may provide escapecontrols such as features, functions, or exit controls that result indeparting from FIG. 8 and performing other operations. For example, itis possible that after one iteration of the process of FIG. 8, there maybe no input at block 808 and therefore the process may suspend orterminate until such time as scrolling input is received.

As a result, a progressive approach is provided in which rapid scrollingof rows of items does not mandate rendering and displaying those itemsbut instead placeholder images, or no images, may be displayed untilsuch time as a pause in scrolling occurs; in response to detecting thepause, images for items are then displayed, with any downloading,decoding or rendering operations then performed as needed. Varioustechniques may be used to detect a pause in input that indicatesscrolling, based upon the hardware platform of client computer 110,operating system 118, or other machine or software characteristics. Inone embodiment, the gallery display logic 128 may subscribe to KEY DOWNand KEY UP events that are published by the operating system 118 orcomponents such as device drivers, and the gallery display logic maydetermine whether a time interval between successive KEY DOWN and KEY UPevents is long enough to indicate a pause. Typical pause time might be0.5 seconds, but may vary in different embodiments and the particulartime used to detect a pause in scrolling is not critical.

The specified number of images may be a configurable value that may varyaccording to the hardware platform on which client computer 110 isbased; an example specified value is fifteen but other embodiments orplatforms may use smaller values such as ten or higher values accordingto the anticipated or known performance characteristics of the hardwareplatform. In an embodiment, the application 114 may define and storemetadata for tiers of performance classes and each of the performanceclasses may store a particular value for the number of graphical imagesto load when a pause occurs.

It will be apparent that the present approach defers performingcomputationally expensive operations for graphical images until a pausein scrolling input occurs. This approach has the benefit of efficientlysupporting applications in which user action to scroll the display maycause movement across the screen of dozens or hundreds of rows of imagesthat are never within the viewport 606 long enough to be seen or focusedupon by the user. In other words, using conventional approaches, thecomputational time needed to render many rows of images that moverapidly through the viewport, either as those images enter the viewportor on a look-ahead basis just before those images enter the viewport,would be wasted. Instead, with the present approaches, networktransactions and GPU operations are performed only when the user haspaused in scrolling and thus indicated an interest in actually seeingthe images rather than merely moving them across the screen withoutviewing them. Still another benefit of the present approach is reductionin GPU and/or CPU contention that would otherwise arise if a largenumber of network transactions were required to download images forimage elements that are only transiently within the viewport 606.

While FIG. 8 and the preceding description has involved verticalscrolling of entire sets of rows, the same process steps may be usedwhen input indicates or requests the horizontal scrolling of a singlerow, or a group of rows. In some embodiments, for the purpose ofensuring a good user experience, the pre-loaded image elements 604 ofFIG. 6 includes a relatively small number of rows but each row isdefined to have a relatively large number of image elements, such asfifteen to fifty image elements. In other words, the application 114 andgallery display logic 128 may be configured to support storing arelatively smaller number of rows in which each row has a relativelylarger number of images.

With this architecture, pre-fetching operations may be used to fetchmetadata for some or all of the non-visible image elements in aparticular row that is visible in viewport 606. For example, for rows A,B of FIG. 6, gallery display logic 128 may be configured to pre-fetchmetadata for all image elements of rows A, B, even those image elementsthat are outside the viewport 606. This process reflects the recognitionthat when a particular row A, B has been paused and is visible in theviewport 606, the user is likely to scroll that row left or rightrelatively slowly, in order to inspect and evaluate many more (or all)images in the row, and therefore pre-fetching all such images willresult in the best user experience by ensuring the ability to rapidlydisplay images as soon as scrolling input occurs. In contrast, verticalscrolling tends to involve rapid operations as users quickly scanthrough numerous categories, such as genres, to find particular rowsthat might contain items of interest.

In an embodiment, the number of image elements that are used to storerendered textures for box art or other graphic image informationtypically is relatively small, and smaller than the total number ofitems in a row for which metadata may be pre-fetched. For example, inone embodiment the media player application 114 could define each row A,B, C, D, E, F as a circular list of 75 items, and could specify whichaudiovisual program item is in each entry of the circular list. For eachrow, the media player application 114 could manage a row structure 620in gallery memory space 602 that contains 75 items as seen in theexample of FIG. 6. The media player application 114 may define ametadata label for the row structure 620, such as “Action/AdventureMovies,” and may determine which items in the media catalog 103 areassociated with that metadata label. Accordingly, the media playerapplication 114 may cause pre-fetching metadata for each audiovisualprogram item for each entry of the row structure 620. Since the metadatatypically comprises a URL pointing to box art, title, and descriptiveinformation, the dataset for the entire row structure 620 is relativelycompact.

Thus in one embodiment, a row structure 620 may be associated with alist of movies or programs. In addition, there may be any number of rowstructures 620 each associated with a different list of movies orprograms, and references to all such row structures may be stored in a“list of list of movies” or LOLOMO that serves as a reference fordetermining which row A, B, C, D, E, F should be displayed next and tolocate metadata relating to that row. Each list in a LOLOMO may vary inlength.

Each row structure 620 is associated with a corresponding row of theimage elements, such as row A in the example of FIG. 6, having fewerimage elements 607, 608, 609 such as ten to twenty elements. With thisstructure, when any part of a particular row A enters the viewport 606,the gallery display logic 128 may cause pre-fetching metadata for imageelements 607, 608, 609 from the row structure 620, and may alsodownload, decode, and render images for those image elements. As aresult, at least ten to twenty images will be capable of immediatedisplay in response to user input indicating a horizontal scrollingoperation. As scrolling operations occur and at least one of the imageelements 608 moves out of the viewport, that non-visible image elementmay be replaced with another downloaded, decoded and rendered image. Inother words, the relatively small number of image elements may beupdated with newly visible—or soon-to-be-visible—graphic images whilethe row structure 620 holds metadata for all audiovisual program itemsthat are associated with a category or other row label.

In one embodiment, the player application 114 may define the viewport606 as larger than the area of a computer screen that is known to beactually visible to a user. For example, the player application 114 mayobtain known dimension data for a particular client computer 110 fromstored device profiles for categories of devices that are maintained atapplication server computer 102, and then the viewport 606 could bedefined as several hundred pixels larger in the vertical and horizontaldimensions than the actual screen of the client computer. For imageelements 607, 609, 611, 612 that lie outside the visible screen area,the player application 114 may set the lazy flag value described hereinto require downloading and decoding when items are in the row structure620 and assigned to one of those image elements; with this approach, theplayer application can cause the gallery display logic 128 and/or imagepre-fetch logic 126 to pre-fetch images for those image elements thatlie outside the visible screen area so that those image elements areready for display in response to a scroll operation. Further, for imageelements in rows C, D, E, F that are far outside the viewport, theplayer application 114 may set the lazy flag value to require either noaction, or downloading, or downloading plus decoding, based upon therelative distance of those rows from the bounds of the viewport.

The techniques described herein may be applied in the context of agraphical user interface of the type seen in FIG. 9. In this example, ascreen display 902 comprises a metadata region 904 having text metadata906 and a billboard image 905. The text metadata 906 may include title,rating, season, summary, and other information about a particularaudiovisual program, and the billboard image 905 may relate to thatitem. Screen display 902 further comprises two (2) horizontally andvertically scrollable rows 908, 910 each comprising a plurality of boxart images 912, 914 that are displayed as consistently sized polygonswithin a row. Each row 908, 910 is associated with a category label,such as BECAUSE YOU WATCHED ORANGE IS THE NEW BLACK or BECAUSE YOUWATCHED RED LIGHTS. Any other appropriate category labels for lists,genres or other categories may be used and those in FIG. 9 are merelyexamples.

In an embodiment, box art image 912 is selected as indicated by adistinctive border, and the metadata region 904 displays data thatcorresponds to the selected box art image. All other box art images inrow 908 may be scrolled left or right as a unit in response to inputindicating lateral movement in a particular horizontal direction.Similarly, rows 908, 910 may be vertically scrolled in the display,without affecting the size or appearance of metadata region 904, inresponse to input indicating lateral movement in a particular verticaldirection.

Box art image 914 comprises a placeholder. For purposes of illustratingan example, assume that row 910 was recently rapidly scrolled into viewin response to user input; in this situation, the application 114 mayhave deferred rendering box art image 914 according to the techniques ofFIG. 8, but in response to a pause that causes row 910 to rest in thevisible region of the screen, box art image 914 will be rendered or, ifneeded, downloaded and decoded.

In some embodiments, the hardware platform associated with clientcomputer 110 may support scrolling operations in which a single inputsignal may indicate scrolling by a plurality of items. For example, withsome hardware platforms, a single left-cursor signal or left-movementsignal might actually signify moving left by four (4) image elementsrather than just one. The embodiments described thus far effectivelysupport operations of these hardware platforms by ensuring that asufficient number of image elements have been pre-fetched for a row tosupport a multiple-element lateral scroll operation; in other words,enough image elements are ready and rendered to permit effective use ofa multiple-element operation.

In an embodiment, the number of image elements 607, 608, 609 topre-fetch for a particular row A may be determined based upon heuristicsor dynamically changed configurable values that are based uponobservations of actions of a plurality of client computers 110, 130, 140or one particular client computer. For example, the player application114 and/or the gallery display logic 128 may determine that users mostcommonly scroll a particular row A, B to the left in order to see imageelements 609, 612 that are offscreen to the right, rather than firstscrolling to the right or regularly scrolling to the right. Based onsuch observations, the player application 114 and/or the gallery displaylogic 128 may establish configuration data that requires pre-fetching alarger number of image elements for the right ends of rows A, B ratherthan the left ends, or may define the viewport 606 in a manner that isbiased toward one end of the rows A, B, C, D, E, F. Feedback signalsindicating user actions at client computers 110, 130, 140 duringinteractions with the player application 114 may be obtained fromoperating system primitives of the operating system 118.

6. Example Implementation Mechanism—Hardware Overview

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 10 is a block diagram that illustrates a computersystem 1000 upon which an embodiment of the invention may beimplemented. Computer system 1000 includes a bus 1002 or othercommunication mechanism for communicating information, and a hardwareprocessor 1004 coupled with bus 1002 for processing information.Hardware processor 1004 may be, for example, a general purposemicroprocessor.

Computer system 1000 also includes a main memory 1006, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 1002for storing information and instructions to be executed by processor1004. Main memory 1006 also may be used for storing temporary variablesor other intermediate information during execution of instructions to beexecuted by processor 1004. Such instructions, when stored innon-transitory storage media accessible to processor 1004, rendercomputer system 1000 into a special-purpose machine that is customizedto perform the operations specified in the instructions.

Computer system 1000 further includes a read only memory (ROM) 1008 orother static storage device coupled to bus 1002 for storing staticinformation and instructions for processor 1004. A storage device 1010,such as a magnetic disk or optical disk, is provided and coupled to bus1002 for storing information and instructions.

Computer system 1000 may be coupled via bus 1002 to a display 1012, suchas a cathode ray tube (CRT), for displaying information to a computeruser. An input device 1014, including alphanumeric and other keys, iscoupled to bus 1002 for communicating information and command selectionsto processor 1004. Another type of user input device is cursor control1016, such as a mouse, a trackball, or cursor direction keys forcommunicating direction information and command selections to processor1004 and for controlling cursor movement on display 1012. This inputdevice typically has two degrees of freedom in two axes, a first axis(e.g., x) and a second axis (e.g., y), that allows the device to specifypositions in a plane.

Computer system 1000 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 1000 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 1000 in response to processor 1004 executing one or moresequences of one or more instructions contained in main memory 1006.Such instructions may be read into main memory 1006 from another storagemedium, such as storage device 1010. Execution of the sequences ofinstructions contained in main memory 1006 causes processor 1004 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperation in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 1010.Volatile media includes dynamic memory, such as main memory 1006. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, and EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 1002. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 1004 for execution. Forexample, the instructions may initially be carried on a magnetic disk orsolid state drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 1000 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 1002. Bus 1002 carries the data tomain memory 1006, from which processor 1004 retrieves and executes theinstructions. The instructions received by main memory 1006 mayoptionally be stored on storage device 1010 either before or afterexecution by processor 1004.

Computer system 1000 also includes a communication interface 1018coupled to bus 1002. Communication interface 1018 provides a two-waydata communication coupling to a network link 1020 that is connected toa local network 1022. For example, communication interface 1018 may bean integrated services digital network (ISDN) card, cable modem,satellite modem, or a modem to provide a data communication connectionto a corresponding type of telephone line. As another example,communication interface 1018 may be a local area network (LAN) card toprovide a data communication connection to a compatible LAN. Wirelesslinks may also be implemented. In any such implementation, communicationinterface 1018 sends and receives electrical, electromagnetic or opticalsignals that carry digital data streams representing various types ofinformation.

Network link 1020 typically provides data communication through one ormore networks to other data devices. For example, network link 1020 mayprovide a connection through local network 1022 to a host computer 1024or to data equipment operated by an Internet Service Provider (ISP)1026. ISP 1026 in turn provides data communication services through theworld wide packet data communication network now commonly referred to asthe “Internet” 1028. Local network 1022 and Internet 1028 both useelectrical, electromagnetic or optical signals that carry digital datastreams. The signals through the various networks and the signals onnetwork link 1020 and through communication interface 1018, which carrythe digital data to and from computer system 1000, are example forms oftransmission media.

Computer system 1000 can send messages and receive data, includingprogram code, through the network(s), network link 1020 andcommunication interface 1018. In the Internet example, a server 1030might transmit a requested code for an application program throughInternet 1028, ISP 1026, local network 1022 and communication interface1018.

The received code may be executed by processor 1004 as it is received,and/or stored in storage device 1010, or other non-volatile storage forlater execution.

Embodiments further include the subject matter of the following numberedclauses:

38. A computer system providing an improvement in efficient use ofcomputer memory and comprising: a first computer comprising one or morenon-transitory computer-readable data storage media stored with one ormore sequences of instructions that are arranged, when executed using asecond computer among a plurality of other computers, to cause thesecond computer to perform: using a computer, creating in computermemory a glyph memory area that is configured to store a plurality ofcached glyphs; using the computer, receiving a request from anapplication to use a particular glyph; in response to the request,determining whether the particular glyph is in the glyph memory area; inresponse to determining that the particular glyph is not in the glyphmemory area: attempting to store a bitmap of the particular glyph to anext location in the glyph memory area; in response to determining thatthe next location is not available a first time, reclaiming space in theglyph memory area in an amount sufficient to store the bitmap;attempting a second time to store the bitmap in the next location in theglyph memory area; in response to determining that the next location isnot available a second time, clearing the glyph memory area of allpreviously stored glyphs and storing the bitmap in the glyph memoryarea; in the first computer, stored downloading instructions which, whenexecuted using the first computer, cause downloading a copy of the oneor more sequences of instructions to any of the other computers.

39. The computer system of clause 38, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform, after determining that the next location isnot available the second time, clearing the glyph memory area andstoring the bitmap in the glyph memory area, performing one or moregraphics operations using the particular glyph that relate to causingdisplaying the particular glyph on a computer display screen of thecomputer.

40. The computer system of clauses 39, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform, performing a single GPU function call thatspecifies a single source to use in a plurality of operations withmultiple glyphs that are in the glyph memory area.

41. The computer system of clause 38, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform, creating, in the computer memory, geometrydata comprising a plurality of glyph items, wherein each of the glyphitems comprises a plurality of metadata values that define a specificglyph and is keyed to a key value; a usage index coupled to the glyphmemory area and configured to store usage values indicating which partsof the glyph memory area are in use.

42. The computer system of clause 38, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform, testing whether the glyph memory area canaccept the particular glyph by comparing a width value and a heightvalue of the particular glyph to other width values and other heightvalues of two or more other glyphs that were most recently added to theglyph memory area.

43. The computer system of clause 38 wherein the determining,attempting, reclaiming, clearing and storing are performed only inresponse to receiving the request and the glyph memory area isconstructed and maintained dynamically in response to requests ofapplications to display glyphs and not in advance of all such requests.

44. A computer system providing an improvement in efficient use ofcomputer memory and comprising: a first computer comprising one or morenon-transitory computer-readable data storage media stored with one ormore sequences of instructions that are arranged, when executed using asecond computer among a plurality of other computers, to cause thesecond computer to perform: using a computer, creating in computermemory a memory space that is configured to store a plurality of visualcontent items in one or more atlases of visual content items, each atlashaving a configured memory size; using the computer, declaring aparticular plurality of visual content items from the plurality ofvisual content items a part of a particular atlas of the one or moreatlases; using the computer, receiving a request from an application todisplay a row of visual content items on a screen of the computer; usingthe computer, determining that the row of visual content items includesone or more visual content items of the particular atlas; using thecomputer, in response to receiving the request, performing a visualcontent processing operation by accessing, in a batch, a plurality ofbitmaps corresponding to the particular atlas.

45. The computer system of clause 44, wherein the visual contentprocessing operation includes writing in a batch the plurality ofbitmaps to the screen of the computer.

46. The computer system of clause 44, wherein the particular atlascorresponds to visible visual content items of the row of visual contentitems displayed on the screen of the computer.

47. The computer system of clause 46, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform: receiving an indication from the applicationthat an additional visual content item is requested to be visible in therow of visual content items; in response to receiving an indication,determining whether assigning the additional visual content item to theparticular atlas would exceed a configured size of the particular atlas;in response to determining that assigning the additional visual contentitem to the particular atlas would not exceed the configured size of theparticular atlas, assigning the additional visual content item to theparticular atlas.

48. The computer system of clause 44, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform: maintaining, as part of the memory space, aleast recently used (LRU) list of visual content items, each entry inthe LRU list ordered relative to other entries in the LRU list based ona contemporariness of last usage of a corresponding visual content itemof said each entry and storing at least metadata for the correspondingvisual content item that includes a network location of thecorresponding visual content item; in response to receiving a requestfrom the application for a visual content item, determining whether anetwork location specified in the request matches a network location ofa visual content item in the LRU list; in response to determining thatthe network location specified in the request matches a network locationof a specific visual content item in the LRU list: satisfying therequest from the application for the visual content item using aspecific LRU entry corresponding to the specific visual content item,and updating a usage flag in metadata of the specific visual contentitem to indicate that the specific content item is in use by theapplication.

49. The computer system of clause 48, wherein the one or more sequencesof instructions comprise instructions that are arranged to cause thesecond computer to perform, in response to determining that the networklocation specified in the request matches the network location of thespecific visual content item in the LRU list: determining that thespecific visual content item is non-visible on the screen of thecomputer, and the request for the visual content item requests thespecific visual content item to be visible on the screen of thecomputer; based on metadata of the specific visual content item,performing one or more of: downloading, decoding and rendering of thespecific visual content item.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense. The sole and exclusive indicator of the scope of the invention,and what is intended by the applicants to be the scope of the invention,is the literal and equivalent scope of the set of claims that issue fromthis application, in the specific form in which such claims issue,including any subsequent correction.

What is claimed is:
 1. A data processing method providing an improvementin efficient use of computer memory and comprising: using a computer,creating in computer memory a glyph memory area that is configured tostore a plurality of cached glyphs; using the computer, receiving arequest from an application to use a particular glyph; in response tothe request, determining whether the particular glyph is in the glyphmemory area; in response to determining that the particular glyph is notin the glyph memory area: attempting to store a bitmap of the particularglyph to a next location in the glyph memory area; in response todetermining that the next location is not available a first time,reclaiming space in the glyph memory area in an amount sufficient tostore the bitmap; attempting a second time to store the bitmap in thenext location in the glyph memory area; in response to determining thatthe next location is not available a second time, clearing the glyphmemory area of all previously stored glyphs and storing the bitmap inthe glyph memory area.
 2. The method of claim 1, further comprising,after receiving the request from the application to use the particularglyph and determining that the particular glyph is not in the glyphmemory area, transforming glyph vector data for the glyph to the bitmap.3. The method of claim 1, further comprising, after determining that thenext location is not available the second time, clearing the glyphmemory area and storing the bitmap in the glyph memory area, performingone or more graphics operations using the particular glyph that relateto causing displaying the particular glyph on a computer display screenof the computer.
 4. The method of claim 3, further comprising performinga single GPU function call that specifies a single source to use in aplurality of operations with multiple glyphs that are in the glyphmemory area.
 5. The method of claim 1, further comprising creating, inthe computer memory, geometry data comprising a plurality of glyphitems, wherein each of the glyph items comprises a plurality of metadatavalues that define a specific glyph and is keyed to a key value; a usageindex coupled to the glyph memory area and configured to store usagevalues indicating which parts of the glyph memory area are in use. 6.The method of claim 5, further comprising creating, in the computermemory, a vertex buffer that is coupled to glyph caching logic that iscoupled to the glyph memory area and the geometry data, wherein thevertex buffer comprises a plurality of location data for glyphs that areexpected to be reused.
 7. The method of claim 1, wherein one or morespecific cached glyphs among the plurality of cached glyphs occupies aregion of the glyph memory area having a different size.
 8. The methodof claim 1, further comprising testing whether the glyph memory area canaccept the particular glyph by comparing a width value and a heightvalue of the particular glyph to other width values and other heightvalues of two or more other glyphs that were most recently added to theglyph memory area.
 9. The method of claim 8 wherein the testing includesleaving a 1-pixel gap between adjacent glyphs.
 10. The method of claim1, wherein the reclaiming comprises performing defragmentation of theglyph memory area by rewriting a plurality of glyphs into new positionsin the glyph memory area that have become available after removal ofunused glyphs.
 11. The method of claim 1, further comprising determiningthat the particular glyph is to be rendered using a visible outline, andin response, storing the particular glyph, associated with a character,in a position in the glyph memory area that is separate from anotherglyph associated with an outline of the character.
 12. The method ofclaim 1 wherein the determining, attempting, reclaiming, clearing andstoring are performed only in response to receiving the request and theglyph memory area is constructed and maintained dynamically in responseto requests of applications to display glyphs and not in advance of allsuch requests.
 13. A data processing method providing an improvement inefficient use of computer memory and comprising: using a computer,creating in computer memory a memory space that is configured to store aplurality of visual content items in one or more atlases of visualcontent items, each atlas having a configured memory size; using thecomputer, declaring a particular plurality of visual content items fromthe plurality of visual content items a part of a particular atlas ofthe one or more atlases; using the computer, receiving a request from anapplication to display a row of visual content items on a screen of thecomputer; using the computer, determining that the row of visual contentitems includes one or more visual content items of the particular atlas;using the computer, in response to receiving the request, performing avisual content processing operation by accessing, in a batch, aplurality of bitmaps corresponding to the particular atlas.
 14. Themethod of claim 13, wherein the visual content processing operationincludes writing in a batch the plurality of bitmaps to the screen ofthe computer.
 15. The method of claim 13, wherein receiving the requestfrom the application to display the row of visual content items on thescreen of the computer is in response to receiving a scrolling input ofa visual row of box art images, on a graphical user interface of theapplication, that correspond to the row of visual content items, andwherein performing the visual content processing operation to displaythe row of visual content items comprises copying, in a batch, theplurality of bitmaps.
 16. The method of claim 13, wherein the particularatlas corresponds to visible visual content items of the row of visualcontent items displayed on the screen of the computer.
 17. The method ofclaim 16, further comprising: receiving an indication from theapplication that an additional visual content item is requested to bevisible in the row of visual content items; in response to receiving anindication, determining whether assigning the additional visual contentitem to the particular atlas would exceed a configured size of theparticular atlas; in response to determining that assigning theadditional visual content item to the particular atlas would not exceedthe configured size of the particular atlas, assigning the additionalvisual content item to the particular atlas.
 18. The method of claim 16,further comprising: receiving an indication from the application that anadditional visual content item is requested to be visible in the row ofvisual content items; in response to receiving an indication,determining whether assigning the additional visual content item to theparticular atlas would exceed a configured size of the particular atlas;in response to determining that assigning the additional visual contentitem to the particular atlas would not exceed the configured size of theparticular atlas, declaring a new atlas with a new memory size, andassigning the additional visual content item to the new atlas.
 19. Themethod of claim 16, further comprising: determining that a memory sizeof a currently non-visible portion of visual content items of theparticular atlas exceeds a particular threshold; in response todetermining that the non-visible portion exceeds the particularthreshold, purging the particular atlas, declaring visible visualcontent items of the particular plurality of visual content items aspart of a new atlas, and reclaiming memory associated with theparticular atlas.
 20. The method of claim 16, further comprising:receiving a request from the application to display another visualcontent item for another row of visual content items corresponding toanother atlas of the one or more atlases; detecting that the othercontent item is the same as a particular visual content item of theparticular atlas; in response to detecting modifying metadata of theother visual content item to reference a bitmap of the particular visualcontent item.
 21. The method of claim 13, further comprising:maintaining, as part of the memory space, a least recently used (LRU)list of visual content items, each entry in the LRU list orderedrelative to other entries in the LRU list based on a contemporariness oflast usage of a corresponding visual content item of said each entry andstoring at least metadata for the corresponding visual content item thatincludes a network location of the corresponding visual content item; inresponse to receiving a request from the application for a visualcontent item, determining whether a network location specified in therequest matches a network location of a visual content item in the LRUlist; in response to determining that the network location specified inthe request matches a network location of a specific visual content itemin the LRU list: satisfying the request from the application for thevisual content item using a specific LRU entry corresponding to thespecific visual content item, and updating a usage flag in metadata ofthe specific visual content item to indicate that the specific contentitem is in use by the application.
 22. The method of claim 21, furthercomprising, in response to determining that the network locationspecified in the request matches the network location of the specificvisual content item in the LRU list, reordering the LRU list to indicatethat the specific LRU entry has been recently used.
 23. The method ofclaim 21, further comprising, in response to determining that thenetwork location specified in the request matches the network locationof the specific visual content item in the LRU list: determining thatthe specific visual content item is non-visible on the screen of thecomputer, and the request for the visual content item requests thespecific visual content item to be visible on the screen of thecomputer; based on metadata of the specific visual content item,performing one or more of: downloading, decoding and rendering of thespecific visual content item.
 24. The method of claim 21, furthercomprising: in response to determining that the network locationspecified in the request does not match any network location in the LRUlist: determining whether allocated memory of the LRU list containsspace for a new visual content item referenced by the network locationspecified in the request; in response to determining that the allocatedmemory of the LRU list does not contain space for the new visual contentitem, removing one or more entries for one or more visual content itemsfrom the LRU list; wherein ordering of the one or more entries indicatesthat the one or more visual content items are less recently used thanother visual content items in the LRU list; wherein total dimensions ofthe one or more visual content items are greater or equal to a dimensionof the new visual content item.
 25. The method of claim 21, furthercomprising: in response to determining that the network locationspecified in the request does not match any network location in the LRUlist: determining whether allocated memory of the LRU list containsspace for a new visual content item referenced by the network locationspecified in the request; in response to determining that the allocatedmemory of the LRU list contains space for the new visual content item:determining that the request for the visual content item requests thenew visual content item to be non-visible on the screen of the computer,and based on metadata of the new visual content item performing one ormore of: downloading, decoding and rendering of the new visual contentitem, when the new visual content item is non-visible on the screen ofthe computer.
 26. A data processing method providing an improvement inefficient use of computer memory and comprising: using a first computer,obtaining from one or more non-transitory computer-readable data storagemedia a copy of one or more sequences of instructions that are stored onthe media and are arranged, when executed using a second computer amonga plurality of other computers to cause the second computer to perform:using a computer, creating in computer memory a glyph memory area thatis configured to store a plurality of cached glyphs; using the computer,receiving a request from an application to use a particular glyph; inresponse to the request, determining whether the particular glyph is inthe glyph memory area; in response to determining that the particularglyph is not in the glyph memory area: attempting to store a bitmap ofthe particular glyph to a next location in the glyph memory area; inresponse to determining that the next location is not available a firsttime, reclaiming space in the glyph memory area in an amount sufficientto store the bitmap; attempting a second time to store the bitmap in thenext location in the glyph memory area; in response to determining thatthe next location is not available a second time, clearing the glyphmemory area of all previously stored glyphs and storing the bitmap inthe glyph memory area; using the first computer, downloading the copy ofthe one or more sequences of instructions to any of the other computers.27. The method of claim 26, further comprising, after determining thatthe next location is not available the second time, clearing the glyphmemory area and storing the bitmap in the glyph memory area, performingone or more graphics operations using the particular glyph that relateto causing displaying the particular glyph on a computer display screenof the computer.
 28. The method of claim 27, further comprisingperforming a single GPU function call that specifies a single source touse in a plurality of operations with multiple glyphs that are in theglyph memory area.
 29. The method of claim 26, further comprisingcreating, in the computer memory, geometry data comprising a pluralityof glyph items, wherein each of the glyph items comprises a plurality ofmetadata values that define a specific glyph and is keyed to a keyvalue; a usage index coupled to the glyph memory area and configured tostore usage values indicating which parts of the glyph memory area arein use.
 30. The method of claim 26, further comprising testing whetherthe glyph memory area can accept the particular glyph by comparing awidth value and a height value of the particular glyph to other widthvalues and other height values of two or more other glyphs that weremost recently added to the glyph memory area.
 31. The method of claim26, wherein the determining, attempting, reclaiming, clearing andstoring are performed only in response to receiving the request and theglyph memory area is constructed and maintained dynamically in responseto requests of applications to display glyphs and not in advance of allsuch requests.
 32. A data processing method providing an improvement inefficient use of computer memory and comprising: using a first computer,obtaining from one or more non-transitory computer-readable data storagemedia a copy of one or more sequences of instructions that are stored onthe media and are arranged, when executed using a second computer amonga plurality of other computers to cause the second computer to perform:using a computer, creating in computer memory a memory space that isconfigured to store a plurality of visual content items in one or moreatlases of visual content items, each atlas having a configured memorysize; using the computer, declaring a particular plurality of visualcontent items from the plurality of visual content items a part of aparticular atlas of the one or more atlases; using the computer,receiving a request from an application to display a row of visualcontent items on a screen of the computer; using the computer,determining that the row of visual content items includes one or morevisual content items of the particular atlas; using the computer, inresponse to receiving the request, performing a visual contentprocessing operation by accessing, in a batch, a plurality of bitmapscorresponding to the particular atlas.
 33. The method of claim 32,wherein the visual content processing operation includes writing in abatch the plurality of bitmaps to the screen of the computer.
 34. Themethod of claim 32, wherein the particular atlas corresponds to visiblevisual content items of the row of visual content items displayed on thescreen of the computer.
 35. The method of claim 34, further comprising:receiving an indication from the application that an additional visualcontent item is requested to be visible in the row of visual contentitems; in response to receiving an indication, determining whetherassigning the additional visual content item to the particular atlaswould exceed a configured size of the particular atlas; in response todetermining that assigning the additional visual content item to theparticular atlas would not exceed the configured size of the particularatlas, assigning the additional visual content item to the particularatlas.
 36. The method of claim 32, further comprising: maintaining, aspart of the memory space, a least recently used (LRU) list of visualcontent items, each entry in the LRU list ordered relative to otherentries in the LRU list based on a contemporariness of last usage of acorresponding visual content item of said each entry and storing atleast metadata for the corresponding visual content item that includes anetwork location of the corresponding visual content item; in responseto receiving a request from the application for a visual content item,determining whether a network location specified in the request matchesa network location of a visual content item in the LRU list; in responseto determining that the network location specified in the requestmatches a network location of a specific visual content item in the LRUlist: satisfying the request from the application for the visual contentitem using a specific LRU entry corresponding to the specific visualcontent item, and updating a usage flag in metadata of the specificvisual content item to indicate that the specific content item is in useby the application.
 37. The method of claim 36, further comprising, inresponse to determining that the network location specified in therequest matches the network location of the specific visual content itemin the LRU list: determining that the specific visual content item isnon-visible on the screen of the computer, and the request for thevisual content item requests the specific visual content item to bevisible on the screen of the computer; based on metadata of the specificvisual content item, performing one or more of: downloading, decodingand rendering of the specific visual content item.